Semi-Automatic Wrapper Generation and Adaption: Living with Heterogeneity in a Market Environment

The success of the Internet as a medium for the supply and commerce of various kinds of goods and services leads to a fast growing number of autonomous and heterogeneous providers that offer and sell goods and services electronically. The new market structures have already entered all kinds of markets. Approaches for market infrastructures usually try to cope with the heterogeneity of the providers by special wrapper components, which translate between the native protocols of the providers and the protocol of the market infrastructure. Enforcing a special interface to the provider limits their independence. Moreover, requirements such as a direct access to the internal business logic and databases of the providers or fix templates for internal data structures are not suitable to establish a real open electronic market. A solution is the limitation of the access to the existing Web interface of the provider. This solution keeps the independence of the providers without burdening them additional work. However, for efficiency reasons, it keeps necessary to tailor a wrapper for each provider. What comes more, each change in the provider or its Web representation forces the modification of the existing wrapper or even the development of a new wrapper. In this paper, we present an approach for a wrapper for complex Web interfaces, which can easily be adapted to any provider just by adding a source description file. A tool allows the construction and modification of source descriptions without expert knowledge. Common changes in the Web representation can be detected and comprehended automatically. The presented approach has been applied to the market of scientific literature.